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I. Real Party in Interest 

The real party in interest in the present appeal is Panasonic Corporation, 
assignee of the present application. 

II. Related Appeals and Interferences 

Appellants, Appellants' undersigned representative, and/or the assignee of the 
present application are unaware of any prior or pending appeals, interferences or 
judicial proceedings which may be related to, directly affect or be directly affected by, or 
have bearing on the Board's decision in the pending appeal. 

III. Status of Claims 

Claims 1, 4-9 and 12-16 are in the instant proceeding and are pending in the 
application. Claims 2-3 and 10-11 have been canceled. Claims 1, 4-9 and 12-16 stand 
finally rejected and are the subject of this appeal. 

IV. Status of Amendments 

No amendments to the claims or specification have been made subsequent to 
the final rejection contained in the Final Office Action dated January 3, 201 1 . 

V. Summary of the Claimed Subject Matter 

Claims 1 and 9 represent the pending independent claims. 
Claim 1: 

Independent claim 1 recites a data processor (100) (application at page 33, lines 
14-20; Fig. 7) for playing back video and audio from a data stream (TS) including video 
data (V) and audio data (A) (application at page 35, lines 4-12; Fig. 8), each of the 
video and audio data being provided with time information (VPTS, APTS) representing 
its presentation time (application at page 3, line 19 to page 4, line 3; page 30, lines 11- 
14; page 41, lines 13-20; Fig. 1(c)), the data processor (100) comprising: 
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a stream acquiring section (1 01 ) for acquiring a first data stream (TS1 ) and a 
second data stream (TS2) continuously (application at page 35, lines 6-12; Figs. 7-8); 

an inserting section (102) for inserting boundary-setting dummy data (71) into a 
data location where the first (TS1 ) and second (TS2) data streams switch each other 
(application at page 36, lines 1-11; Figs. 7-8); 

an analyzing section (103), which detects the dummy data (71), assigns different 
pieces of identification information (nv.na) to the first and second data streams 
(TS1,TS2), and associates the identification information (nv,na) with the video and 
audio data of each said data stream (application at page 38, line 6 to page 39, line 18; 
Figs. 15-16); 

a control section (118) for controlling the respective output timings of video 
represented by video data and audio represented by audio data by reference to the 
time information (VPTS) of the video data and the time information (APTS) of the audio 
data that are associated with the same piece of identification information (nv,na) 
(application at page 40, line 8 to page 41 , line 7; Fig. 18); and 

an output section (117,110) for outputting the video and the audio at the output 
timings (application at page 64, lines 4-10), 

wherein the control section (118) finds the respective presentation end times of 
the video and the audio (video_end_PTS, audio_end_PTS) of the first data stream 
(TS1) according to the time information (VPTS) added to the video data and the time 
information added to the audio data (APTS) (application at page 51, lines 4-14; Fig. 9), 
and 

if the presentation end time of the audio (audio_end_PTS) is later than that of 
the video (video_end_PTS), the control section (118) stops outputting the audio from 
the presentation end time of the video (video_end_PTS) through the presentation end 
time of the audio (audio_end_PTS) (application at page 61, line 13 to page 62, line 6; 
Figs. 10 and 18; S402,S405), and 
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wherein the control section (118) finds the respective presentation start times of 
the video and the audio (video_start_PTS, audio_start_PTS) of the second data stream 
(TS2) according to the time information (VPTS) added to the video data and the time 
information (APTS) added to the audio data (application at page 51, lines 4-14; Fig. 9), 
and 

if the presentation start time of the audio (audio_start_PTS) is earlier than that of 
the video (video_start_PTS), the control section (118) stops outputting the audio from 
the presentation start time of the audio (audio_start_PTS) through the presentation start 
time of the video (video_start_PTS) (application at page 44, line 13 to page 45, line 4; 
page 66, line 14 to page 67, line 14; Figs. 11 and 18; S408,S409). 

Claim 9: 

Claim 9 recites a data processing method for playing back video and audio from 
a data stream (TS) including video data (V) and audio data (A) (application at page 35, 
lines 4-12; Fig. 8), each of the video and audio data being provided with time 
information (VPTS.APTS) representing its presentation time (application at page 3, line 
1 9 to page 4, line 3; page 30, lines 11-14; page 41 , lines 1 3-20; Fig. 1 (c)), the method 
comprising the steps of: 

acquiring a first data stream (TS1) and a second data stream (TS2) continuously 
(application at page 35, lines 6-12; Figs. 7-8); 

inserting boundary-setting dummy data (71) into a data location where the first 
(TS1) and second (TS2) data streams switch each other; 

detecting the dummy data (71), assigning different pieces of identification 
information (nv,na) to the first and second data streams (TS1 ,TS2), and associating the 
identification information (nv,na) with the video and audio data of each said data stream 
(application at page 38, line 6 to page 39, line 18; Figs. 15-16); 

controlling the respective output timings of video represented by video data and 
audio represented by audio data by reference to the time information (VPTS) of the 
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video data and the time information (APTS) of the audio data that are associated with 
the same piece of identification information (nv,na) (application at page 40, line 8 to 
page 41 , line 7; Fig. 1 8); and 

outputting the video and the audio at the output timings (application at page 64, 
lines 4-10), 

wherein the step of controlling includes the steps of: 

finding the respective presentation end times of the video and the audio 
(video_end_PTS, audio_end_PTS) of the first data stream (TS1 ) according to the time 
information (VPTS) added to the video data and the time information added to the 
audio data (APTS) (application at page 51 , lines 4-14; Fig. 9), and 

if the presentation end time of the audio (audio_end_PTS) is later than that of 
the video (video_end_PTS), stopping outputting the audio from the presentation end 
time of the video (video_end_PTS) through the presentation end time of the audio 
(audio_end_PTS) (application at page 61 , line 1 3 to page 62, line 6; Figs. 1 0 and 1 8; 
S402,S405), and 

wherein the step of controlling includes the steps of: 

finding the respective presentation start times of the video and the audio 
(video_start_PTS,audio_start_PTS) of the second data stream (TS2) according to the 
time information (VPTS) added to the video data and the time information (APTS) 
added to the audio data (application at page 51 , lines 4-14; Fig. 9), and 

if the presentation start time of the audio (audio_start_PTS) is earlier than that of 
the video (video_start_PTS), stopping outputting the audio from the presentation start 
time of the audio (audio_start_PTS) through the presentation start time of the video 
(video_start_PTS) (application at page 44, line 13 to page 45, line 4; page 66, line 14 to 
page 67, line 14; Figs. 11 and 18; S408.S409). 

VI. Grounds of Rejection to be Reviewed on Appeal 
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Claims 1 , 4-9 and 1 2-1 6 stand rejected pursuant to 35 USC §1 03(a) as being 
obvious over Takamori et ai, U.S. Patent No. 6,041 ,067 (Takamori et al.) in view of 
Suzuki, U.S. Patent No. 5,808,722 {Suzuki). 

VII. Argument 

Claim 1 recites a data processor in which a control section finds the respective 
presentation end times of the video and the audio of the first data stream according to 
the time information added to the video data and the time information added to the 
audio data, and if the presentation end time of the audio is later than that of the video, 
the control section stops outputting the audio from the presentation end time of the 
video through the presentation end time of the audio (application at page 61, line 13 to 
page 62, line 6; Figs. 10 and 18; S402.S405). Moreover, the control section finds the 
respective presentation start times of the video and the audio of the second data 
stream according to the time information added to the video data and the time 
information added to the audio data, and if the presentation start time of the audio is 
earlier than that of the video, the control section stops outputting the audio from the 
presentation start time of the audio through the presentation start time of the video 
(application at page 44, line 13 to page 45, line 4; page 66, line 14 to page 67, line 14; 
Figs. 11 and 18; S408,S409). 

Thus, the data processor of claim 1 stops outputting of the audio in the case 
where the presentation end time of the audio is later than that of the video in the first 
data stream, or in the case where the presentation start time of the audio is earlier than 
that of the video in the second data stream. In other words, the data processor of claim 
1 processes the data under such circumstances not by synchronizing the audio and 
video but rather by stopping the outputting of the audio in the event the presentation 
end time of the audio is later than that of the video in the case of the first data stream, 
or in the event where the presentation start time of the audio is earlier than that of the 
video in the case of the second data stream. 

The Examiner has acknowledged that Takamori et al. does not explicitly teach a 
control section finding the respective presentation end times of the video and the audio 
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of the first data stream according to the time information added to the video data and 
the time information added to the audio data, and wherein if the presentation end time 
of the audio is later than that of the video, the control section stops outputting the audio 
from the presentation end time of the video through the presentation end time of the 
audio as recited in claim 1 . The Examiner similarly has acknowledged that Takamori et 
al. does not explicitly teach the control section stopping outputting the audio from the 
presentation start time of the audio through the presentation start time of the video if 
the presentation start time of the audio is earlier than that of the video as recited in 
claim 1 . [OA dated July 29, 201 0, pgs. 4-5]. 

On the other hand, the Examiner has taken the position that Suzuki makes up for 
the rather sizable deficiencies in Takamori et al., and that it would have been obvious to 
one having ordinary skill to modify the teachings of Takamori et al. based on Suzuki so 
as to result in the claimed invention. As applicants previously argued to the Examiner, 
the rejection should be withdrawn/reversed for at least the following reasons. 

Takamori et al. relates to a device for synchronizing data processing. Even more 
particularly, Takamori et al. teaches: 

It is an object of the present invention to provide a device for synchronizing data 
processing, that can correctly detect boundaries of plural data sequences, each 
sequence comprising a series of data blocks having a coherent content, and that 
can process and output data included in the data sequences successively, in the 
temporal order, without skipping or delaying the processing . (Emphasis Added; 
Col. 6, Ins. 18-24). 

Clearly, the intended purpose of the invention of Takamori et al. is to provide a 
device for synchronizing which does not skip or delay the processing of the data. As 
noted above, the Examiner has admitted that Takamori et al. does not teach such 
stopping of the processing but argues it would have been obvious to modify Takamori 
et al. based on Suzuki in order to do so. 

However, applicants respectfully note that such modification would be directly 
contrary to the objective in Takamori et al. and hence would not be obvious to one 
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having ordinary skill. It is well settled that the proposed modification cannot render the 
prior art unsatisfactory for its intended purpose. [MPEP §21 43.01 (V.)] If the proposed 
modification would render the prior art invention being modified unsatisfactory for its 
intended purpose, then there is no suggestion or motivation to make the proposed 
modification. In re Gordon, 733 F.2d 900, 221 USPQ 1 125 (Fed. Cir. 1984). 

The modification which has been proposed by the Examiner, namely modifying 
the invention of Takamori et al. to stop the outputting of data, would render the 
invention of Takamori et al. unsatisfactory for its expressly intended purpose, namely 
not skipping or delaying the processing. 

Applicants therefore respectfully submit that the modification presented by the 
Examiner would not have been obvious and the rejection of claim 1 should be reversed. 
Similar comments apply with respect to corresponding method claim 9. 

While the above arguments were presented to the Examiner in applicants' 
response to filed on October 27, 2010, they were rejected by the Examiner in the Final 
Office Action mailed on January 3, 201 1 , in which the Examiner stated: 

Response to Arguments 

2. Applicant's arguments filled October 27, 2Q10 have tem fully considered but they 
are not persuasive 

The applicant argues that SuzuW does not teach ° wherein the control section 
finds the respective presentation end times of the video and the audio of the first data 
stream according to the rime information added to the video data and the time 
information added to the audio data, and wherein if She presentation end time of the 
i f < jt- i f ui i 1 ■ < the cot ol secti i t m\ affling the aw 
the presentation end time of the video through the presentation end time of the audio, 
wfiereto She control section finds the rsspe - . of She vkteo and 

the audio of the second data stream according to the time information added tc the 
video daia and the time information added to the audio data, and wherein if the 
presentation start time of the audio is earlier than that of the video, the control section 
t ngtltt r the presentation start time of the audio through the 

presentation start time of the video*. The examiner respectfully disagrees. Suzuki 
t teto 'he I ■= > - afion of the video date and the audio 

data to ensure hai fhe> - synchronized together (column 7 lines 1 - 10). 
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Thus, the Examiner maintained the rejection on the basis that "Suzuki teaches 
that a controller looks to the time information of the video data and the audio data to 
ensure that they are synchronized together (citing Col. 7, Ins. 1 -1 0)." In this regard, 
applicants pointed out to the Examiner that they agreed with the Examiner's 
understanding. Namely, applicants agreed with the Examiner that Suzuki's controller 
looks to the time information of the video data and the audio data to ensure that they 
are synchronized together. 

Suzuki: 




DISPLAY FRAME 



AUOiO OUTPUT 



FIG. 2 



For example, Fig. 2 of Suzuki (reproduced above) illustrates how the actual 
display frame (i.e., the video) and audio output are relatively synchronous. Therefore, 
consistent with the Examiner's and the applicants' understanding, Suzuki does teach 
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that the controller looks to the time information of the video data and the audio data to 
ensure that they are synchronized together. 

However, applicants again respectfully submit that such action in Suzuki is 
contrary to that which is being claimed in claims 1 and 9. Claim 1 recites how if the 
presentation end time of the audio is later than that of the video, the control section 
stops outputtinq the audio from the presentation end time of the video through the 
presentation end time of the audio. Similarly, claim 9 recites the step of stopping 
outputtinq the audio from the presentation end time of the video through the 
presentation end time of the audio if the presentation end time of the audio is later than 
that of the video. 

Suzuki purposely does not stop the audio output, as this enables the controller in 
Suzuki to maintain the video output and audio output relatively synchronous. As shown 
in Fig. 2 of Suzuki and discussed at column 7, lines 1 1-22, for example, audio data is 
output from the first frame having audio PTS "104500". The audio decoder 30 outputs 
the output audio data 31 completely synchronized with the system time clock (STC) 
102. In other words, the output audio signal 31 is synchronized relatively and 
completely with the display video data 51 . (Col. 7, Ins. 1 2-22). The audio output in 
Suzuki is not stopped at all, else the audio output could not be synchronized as taught 
in Suzuki. 

Applicants therefore again note that the data processor/method of claims 1 and 9 
recite stopping outputting of the audio in the case where the presentation end time of 
the audio is later than that of the video, or in the case where the presentation start time 
of the audio is earlier than that of the video. Thus, the data processor/method of claims 
1 and 9 process the data not by synchronizing the audio and video but rather by 
stopping the outputting of the audio in the event the presentation end time of the audio 
is later than that of the video, or the case where the presentation start time of the audio 
is earlier than that of the video. 

The Examiner acknowledged that Takamori et al. does not explicitly teach a 
control section finding the respective presentation end times of the video and the audio 
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of the first data stream according to the time information added to the video data and 
the time information added to the audio data, and wherein if the presentation end time 
of the audio is later than that of the video, the control section stops outputting the audio 
from the presentation end time of the video through the presentation end time of the 
audio. Moreover, for the reasons expressed above Suzuki does not make up for the 
deficiencies in Takamori et al. 

Still further, applicants note that claims 1 and 9 recite the condition of whether 
the presentation start time of the audio is earlier than that of the video. The Examiner 
relied on Suzuki at column 6, line 25 to column 7, line 10 as teaching the condition of 
the presentation start time of the audio being earlier than that of the video. However, 
applicants do not find any such teaching as purported by the Examiner. In fact, in their 
response to the Final Office Action applicants noted that should the Examiner maintain 
such teaching in Suzuki, it was requested that the Examiner point out such teaching in 
Suzuki with more particularity. 

In the Pre-Appeal Brief Conference Request filed by the applicants, it was 
pointed out how applicants had responded to the Final Office action on May 2, 201 1 
without amendment to the claims. Rather the applicants responded with particularity 
what they perceived were the shortcomings in the Examiner's rejection. Applicants 
included in their response detailed analysis, and not merely conclusory statements. 
Their analysis specifically addressed what the Examiner had stated as the basis for the 
rejection. For example, applicants pointed out how: 

(1 ) Suzuki purposely does not stop the audio output, whereas the claimed invention 
stops outputting the audio (Resp., p. 4); and 

(2) The Examiner's reliance on Suzuki as teaching the condition of the presentation 
start time of the audio being earlier than that of the video is not supported at 
column 6, line 25 to column 7, line 10 as alleged by the Examiner. Applicants 
expressly requested that should the Examiner maintain such understanding, the 
Examiner point to where such teaching is in Suzuki with more particularity. 
(Resp., p. 5). 
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Applicants pointed out how in the Advisory Action, the Examiner merely repeated 
the language of the rejection made in the Final Office Action with the exception of a 
single additional sentence, namely: 

The presentation end times are recognized by the values of presentation time 
stamp which taught (sic) by Suzuki. (Advisory Action, sentence bridging pp. 2-3). 

Applicants emphasized that this added statement by the Examiner was merely 
conclusory. Moreover, applicants pointed out that the Examiner's statement appeared 
in no way to be responsive to the noted distinctions argued by the applicants. As 
applicants pointed out to the Pre-Appeal Brief review panel, clearly it is not possible to 
move forward and reach resolution in the prosecution of an application if the applicant 
and examiner are not responsive to each other's arguments. Disappointingly, the panel 
merely indicated that the appeal would proceed to the Board of Appeals and 
Interferences. 

Applicants respectfully request that the Board of Appeals and Interferences 
acknowledge the shortcomings of the rejection and reverse the Examiner's findings. 
Neither Takamori et al. nor Suzuki, whether taken alone or in combination, teach or 
render obvious the invention of claims 1 , 9, and those dependent therefrom. Neither 
Takamori et al. nor Suzuki teach or render obvious the features where if the 
presentation end time of the audio is later than that of the video in the first data stream, 
the control section stops outputting the audio from the presentation end time of the 
video through the presentation end time of the audio; and if the presentation start time 
of the audio is earlier than that of the video in the second data stream, the control 
section stops outputting the audio from the presentation start time of the audio through 
the presentation start time of the video. 

VIII. Claims Appendix 

An appendix containing a copy of the claims involved in this appeal is attached to 
this brief. 
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IX. Evidence Appendix 



An evidence appendix is attached, but identifies no items of evidence. 



X. Related Proceedings Appendix 



A related proceedings appendix is attached, but identifies no decisions. 
Respectfully submitted, 

RENNER, OTTO, BOISSELLE & SKLAR, LLP 

/Mark D. Saralino/ 

Mark D. Saralino 
Reg. No. 34,243 

DATE: September 22, 2011 

The Keith Building 
1621 Euclid Avenue 
Nineteenth Floor 
Cleveland, Ohio 44115 
(216)621-1113 
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CLAIMS APPENDIX 



1 . A data processor for playing back video and audio from a data stream 
including video data and audio data, each of the video and audio data being provided 
with time information representing its presentation time, the data processor comprising: 

a stream acquiring section for acquiring a first data stream and a second data 
stream continuously; 

an inserting section for inserting boundary-setting dummy data into a data 
location where the first and second data streams switch each other; 

an analyzing section, which detects the dummy data, assigns different pieces of 
identification information to the first and second data streams, and associates the 
identification information with the video and audio data of each said data stream; 

a control section for controlling the respective output timings of video 
represented by video data and audio represented by audio data by reference to the 
time information of the video data and the time information of the audio data that are 
associated with the same piece of identification information; and 

an output section for outputting the video and the audio at the output timings, 

wherein the control section finds the respective presentation end times of the 
video and the audio of the first data stream according to the time information added to 
the video data and the time information added to the audio data, and 

if the presentation end time of the audio is later than that of the video, the control 
section stops outputting the audio from the presentation end time of the video through 
the presentation end time of the audio, and 

wherein the control section finds the respective presentation start times of the 
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video and the audio of the second data stream according to the time information added 
to the video data and the time information added to the audio data, and 

if the presentation start time of the audio is earlier than that of the video, the 
control section stops outputting the audio from the presentation start time of the audio 
through the presentation start time of the video. 

4. The data processor of claim 1 , wherein when finding given video data and 
audio data associated with different pieces of identification information, the control 
section gets only the video represented by the video data output first, and 

wherein when finding video data, obtained after the video has been played back, 
and the audio data associated with the same piece of identification information, the 
control section controls the output timings of the video represented by the video data 
and the audio represented by the audio data in accordance with the time information of 
the video data and the time information of the audio data that are associated with the 
same piece of identification information. 

5. The data processor of claim 1 , wherein the stream acquiring section is 
able to acquire three or more data streams continuously, and 

wherein the inserting section inserts dummy data, which has monotonically 
increasing or decreasing values corresponding to the identification information, into 
every data location where associated two of the continuously acquired data stream 
switch each other. 
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6. The data processor of claim 5, wherein when finding the piece of 
identification information associated with the audio data agreeing with a piece of 
identification information associated with video data representing video that was output 
in the past, the control section stops outputting audio represented by the audio data 
and starts outputting audio represented by audio data having the same piece of 
identification information as that associated with the video data of the video being 
output currently. 

7. The data processor of claim 1 , wherein when finding the piece of 
identification information associated with the audio data agreeing with a piece of 
identification information associated with video data representing video that has not 
been output yet, the control section stops outputting audio represented by the audio 
data and does not start outputting the audio represented by the audio data until the 
piece of identification information agrees with a piece of identification information 
associated with video data acquired afterward. 

8. The data processor of claim 1 , wherein each said data stream has a 
packet structure including packets that store video data and packets that store audio 
data, and 

wherein the inserting section inserts the boundary-setting dummy packet 
between the last packet of the first data stream and the first packet of the second data 
stream. 
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9. A data processing method for playing back video and audio from a data 
stream including video data and audio data, each of the video and audio data being 
provided with time information representing its presentation time, the method 
comprising the steps of: 

acquiring a first data stream and a second data stream continuously; 

inserting boundary-setting dummy data into a data location where the first and 
second data streams switch each other; 

detecting the dummy data, assigning different pieces of identification information 
to the first and second data streams, and associating the identification information with 
the video and audio data of each said data stream; 

controlling the respective output timings of video represented by video data and 
audio represented by audio data by reference to the time information of the video data 
and the time information of the audio data that are associated with the same piece of 
identification information; and 

outputting the video and the audio at the output timings, 

wherein the step of controlling includes the steps of: 

finding the respective presentation end times of the video and the audio of the 
first data stream according to the time information added to the video data and the time 
information added to the audio data, and 

if the presentation end time of the audio is later than that of the video, stopping 
outputting the audio from the presentation end time of the video through the 
presentation end time of the audio, and 

wherein the step of controlling includes the steps of: 
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finding the respective presentation start times of the video and the audio of the 
second data stream according to the time information added to the video data and the 
time information added to the audio data, and 

if the presentation start time of the audio is earlier than that of the video, 
stopping outputting the audio from the presentation start time of the audio through the 
presentation start time of the video. 

12. The data processing method of claim 9, wherein the step of controlling 
includes the steps of: 

when finding given video data and audio data associated with different pieces of 
identification information, getting only the video represented by the video data output 
first, and 

when finding video data, obtained after the video has been played back, and the 
audio data associated with the same piece of identification information, controlling the 
output timings of the video represented by the video data and the audio represented by 
the audio data in accordance with the time information of the video data and the time 
information of the audio data that are associated with the same piece of identification 
information. 

1 3. The data processing method of claim 9, wherein the step of acquiring 
includes acquiring three or more data streams continuously, and 

wherein the step of inserting includes inserting dummy data, which has 
monotonically increasing or decreasing values corresponding to the identification 
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information, into every data location where associated two of the continuously acquired 
data stream switch each other. 

14. The data processing method of claim 1 3, wherein the step of controlling 
includes the steps of: 

when finding the piece of identification information associated with the audio 
data agreeing with a piece of identification information associated with video data 
representing video that was output in the past, stopping outputting audio represented by 
the audio data and 

starting outputting audio represented by audio data having the same piece of 
identification information as that associated with the video data of the video being 
output currently. 

1 5. The data processing method of claim 9, wherein the step of controlling 
includes the steps of: 

when finding the piece of identification information associated with the audio 
data agreeing with a piece of identification information associated with video data 
representing video that has not been output yet, stopping outputting audio represented 
by the audio data and 

not starting outputting the audio represented by the audio data until the piece of 
identification information agrees with a piece of identification information associated 
with video data acquired afterward. 



6 



Serial No. 10/561,706 

16. The data processing method of claim 9, wherein each said data stream 
has a packet structure including packets that store video data and packets that store 
audio data, and 

wherein the step of inserting includes inserting the boundary-setting dummy 
packet between the last packet of the first data stream and the first packet of the 
second data stream. 
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